Field
The present disclosure relates to an image processing apparatus, an image processing method, and a storage medium.
Description of the Related Art
In general, a technique for performing synchronization imaging based on multiple viewpoints, each of the viewpoints associated with one of a plurality of cameras disposed in different positions, and generating a virtual viewpoint image formed from, in addition to images captured from the viewpoints of the cameras, arbitrary viewpoints generated using a plurality of viewpoint images obtained by the imaging, has attracted attention. Generation and browsing of a virtual viewpoint image based on images obtained from a plurality of viewpoints may be realized by collecting images captured by a plurality of cameras in an image processor, such as a server, performing a process, such as rendering, based on a virtual viewpoint using the image processor, and displaying a virtual viewpoint image on a viewing terminal of a user.
In a service using such a virtual viewpoint image, a user may watch a game of soccer, a basketball, or the like, displayed on a terminal screen as a virtual viewpoint image. Furthermore, if the user is enabled to freely shift a viewpoint of a virtual viewpoint image using a user interface, the user may experience a you-are-there feeling as a more realistic sensation when compared with viewing general captured images. Note that a virtual viewpoint may not only be shifted by a user of a viewing terminal but may also be set by an operation of a user who uses a viewpoint setting device for setting a viewpoint, for example. In this case, a virtual viewpoint image corresponding to a virtual viewpoint set by a user operating the viewpoint setting device is displayed on at least one viewing terminal. Note that when the term “user” is used hereinafter in this disclosure, it should be understood as including both the concepts of a user of a viewing terminal capable of setting a virtual viewpoint and a user of the viewpoint setting device, unless otherwise stated.
Japanese Patent Laid-Open No. 2014-215828 discloses a technique for generating and displaying a virtual viewpoint image corresponding to an arbitrary viewpoint using images within a given range of viewpoints which are captured by a plurality of cameras arranged so as to encompass the given range.
However, in a case where a virtual viewpoint image is viewed while a viewpoint is shifted in accordance with a user operation, when a subject (for example, a player or a ball) which is focused on by the user moves quickly, the subject may move out of a frame of the viewpoint. In particular, as a viewpoint is set closer to a running player or a ball, more realistic content may be obtained. However, the subject may move out of a frame.